Streaming of digital data to a portable device

ABSTRACT

The present invention achieves wireless delivery of media streams to portable devices for providing media information related to a show or attraction for the purposes of captioning, language translation, and assistive listening. As a person moves about a venue, the device receives infrared or radio signals from transmitters at venue locations. The device decodes each signal as venue location, event time, or device trigger for presentation on the device. The device memory contains audio, text, graphics and device playback may contain one or many of these stored contents. The stored content is synchronized with the user&#39;s location, time of travel, time of day, time of a show, or an event in a show and may be in one or many languages. Presentation of stored content on the device may also be altered by user preference, stored logic constraints, automatic learning of user behavior, or user interaction with other devices.

RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional ApplicationSerial No. 60/421,255 entitled “Streaming of Digital Data to a PortableDevice”, filed on Oct. 25, 2002 which is incorporated herein byreference in its entirety.

BACKGROUND

[0002] 1. Field of the Invention

[0003] The present invention relates generally to the use of infraredand/or radio frequencies to stream digital data to portable devices in amass audience or in designated broadcast area.

[0004] 2. Description of the Background Art

[0005] There are several circumstances which may interfere with aperson's ability to hear, and thereby interfere with the presentation ofinformation to the listener. For example, persons with hearing loss maymiss narratives, sound effects, music and other sound material relatedto the presentation, and messages in live performances, films,television and special events. Persons who do not speak the language orlanguages used in the presentation may miss narratives and other relatedmessages that are presented. The language barrier prevents many peoplefrom different cultures and languages from understanding, participatingor interacting with the information being presented. Backgroundenvironmental noise may also affect a person's ability to hear, andthereby diminish the effect of the presentation. Additionally,presentations often could be made more effective if they included amethod to provide interactivity between the audience and thepresenter(s).

[0006] Captioning systems have been used in many venues includingmuseums, theaters and other auditoriums to provide foreign languagetranslation or captioning for the hearing impaired. These systems areeither 1) “open captioning” on a projected surface or large adjacentdisplay area where the entire audience can see the captioning; 2)reflective captioning using a transparent but reflective panel todisplay the text from a rear projection while allowing the viewer to seethe display or performance through the panel; or, 3) hard-wired displaysin the back of the seat in front of the viewer.

[0007] Wireless streaming systems also exist. These consist of modulatedanalog and digitized audio broadcasting systems such as assistivelistening systems and wireless headphones. Also short-range (often 1meter or less) digital transceivers are typically used to exchange databetween computers or between computers and peripherals. These systems donot synchronize multiple units.

SUMMARY OF THE INVENTION

[0008] It is therefore an object of this invention to provide a methodof presenting random and/or synchronized information such as narratives,translations, interactive games, control signal commands or other showrelated messages, to patrons of shows, movie theaters, exhibit halls,auditoriums and/or designated areas through an unobtrusive device.

[0009] The present invention makes use of infrared (IR) emission and/orradio frequency (RF) transmission to stream data to a portable devicefor closed captioning, language translation for multi-cultural languagegroups, previews, games, control of devices and/or similar applications.The portable captioning device could be carried by a person and/or thedevice could be placed in a fixture for hands free use. Before or at thetime of the start of a presentation or presentations, the IR/RF systemwill start the transmission of data to the portable units. Thetransmission will synchronize the portable device with the presentationor presentations for captioning, language translation, previews, games,control of devices and/or similar applications. The portable devicemight be based on existing technologies such as mobile phone, personaldigital assistant (PDA) or a combination of both mobile phone and PDA, acustom designed device specifically for this application, or aninteractive device. This system can be combined with an existing audiostream for the hearing impaired, descriptions for the blind and/orlanguage translation. For example, the invention may provide forinfrared streaming for assistive listening systems.

[0010] Possible user device implementations include a custom portabletext display terminal having a receiver, display controller,micro-controller, small amount of memory, and power source. Similarly, acustom audio unit having a receiver, digital-to-analog converter, audioamplifier, and speaker, could also be used.

[0011] Other devices include talking or motorized toys and games capableof receiving content appropriate to a presentation. Such devices mayalso allow interaction with nearby units. Special effects units havingsome combination of lights, audio effects, or animations could also beused. These units could be held by the user or mounted in thepresentation space as a remotely controlled device. Still other devicesinclude a digital map or way-finding unit, messaging system that allowsthe user to broadcast and receive data to and from nearby units, or useridentification unit that broadcasts user identity, preferences, orlocation to nearby units.

[0012] The present invention provides several benefits over prior artsystems. Prior art systems are not aware of user preferences, such asthe user's language, gender, age, etc. Prior art wireless audio systemssuffer from signal loss and deterioration of the transmitted audio whenthe user moves relative to the infrared transmitter. Existing digitalwireless audio systems do not have provisions for mixing text or controldata in the audio data stream.

[0013] The present invention provides for a combination of features notfound in prior art wireless data communication systems. For example, thesystem and method of the present invention allows multiple users'devices to simultaneously receive identical data and remain synchronizedwith the presentation(s). User devices are able to receive differenttypes of data, such as audio, animation control, text, etc. during asingle transmission. User devices are furthermore able to interact withthe presentation, interact with the presenters, or interact with eachother.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014]FIG. 1 is a schematic view of a system of the invention utilizingIR signals.

[0015]FIG. 2 is a schematic view of a system of the invention utilizingRF signals.

[0016]FIG. 3 is a schematic view of a portable display device receivingan IR signal.

[0017]FIG. 4 is a schematic view of an embodiment of the presentinvention utilizing radio frequency (RF) signals.

[0018]FIG. 5 is an exemplary depiction of a data packet stream deliveredby a high speed transmitter.

[0019]FIG. 6 is an exemplary embodiment of the portable device inaccordance with the present invention.

[0020]FIGS. 7a-c is a flow diagram in accordance with the teachings ofthe present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Reference will now be made in detail to an exemplary embodimentof the present invention, an example which is illustrated in theaccompanying drawings. It is to be understood that other embodiments maybe utilized and structural and functional changes may be made withoutdeparting from the respective scope of the present invention.

[0022] The hardware requirements of the preferred embodiment of thissystem are depicted in FIGS. 1 and 2. A time code reader 10, which inthis embodiment is an optical time code reader, is capable of readingtime codes from a show device 12 such as a film projector, show controlcomputer or another media source. The media source may include analogand/or digital content and could be audio, video, and other information(e.g., force feedback parameters for tactile sensing). A centralprocessing unit (CPU) 14 receives the time code signal from the reader10 and synchronizes the content with the film and/or presentation, whichmay be a movie screen 15. Examples of content are text captioning,language translation, games and/or other related applications. Thecentral processing unit 14 has the capability to access and interpretthe content. The content can reside in the internal memory of thecentral processing unit 14 and/or as a removable memory media.

[0023] The CPU 14 will deliver the synchronized data to the infrared(IR) emitters 16, which are capable of delivering IR messages or controldata to a portable device 18, which may be operated either in an indooror outdoor environment. Alternatively, a low-powered licensed and/ornon-licensed radio frequency (RF) system can also be used to deliver thesynchronized data to the portable device via an RF signal. FIG. 2depicts an RF receiver 24 and RF transmitter 22, which interact with aportable RF capable device 18 to achieve the same results as the IRsystem previously described.

[0024] The portable display device 18 has sufficient memory to allowstorage of all data to be presented. All required data may betransmitted to the device 18 in one session for display at predeterminedtimes after receipt of a start signal, or the data may be transmitted ina plurality of sessions for display in real time. The stored data may bedisplayed in its entirety, or a portion of the data may be displayed.The device 18 includes infrared ports capable of receiving and/oremitting infrared messages. Messages emitted from the portable displaydevice 18 are read by one or more IR detectors 19. In an alternativeembodiment, the device 18 could also contain RF receiver and/ortransmitter ports capable of receiving and transmitting RF messages. Theportable device 18 will receive the IR or RF signal and convert thesignal to information that can be stored and/or displayed in sync withthe presentation. The device 18 may also contain the capability toreceive and play audio such as for assistive listening and/or audiolanguage translations, or program material specific to the presentation.

[0025] The system could also recognize show/presentation start and/orend signals. The system could then transmit random and/or synchronizedinformation to the patrons possessing a device. This will allow thepatrons to interact with the device while waiting for theshow/presentation to start or after the show/presentation has ended.

[0026] One possible application of the invention is depicted in FIG. 3.In this application, time code information is read by the CPU 14, whichthen accesses on board content, and delivers the synchronized data tothe infrared emitters 16, which are in turn capable of delivering IRmessages. A portable device 24 receives the IR messages and converts theIR messages to presentable data. In this instance, the IR receiver 19 isa modification to an existing device such as a PDA (for example, a Palmtype device) and/or a pocket PC (for example, a Compaq iPAQ) that canstore and/or immediately display the data. The IR receiver takes the IRsignal from the emitter and translates it to an electronic signal forthe serial port 22 of the PDA and/or pocket PC. A terminal softwareprogram converts the electronic signal into data that is presented astext on the display screen 26.

[0027] This invention may be adapted for uses such as text captioningand language translation in movie or live theaters, consumer productswhich can provide an interactive experience, and to provide a wirelesslink for control signals to equipment, devices or products which areused in public presentations.

[0028] Another application of the invention is depicted in FIG. 4. Inthis application, a combination of RF transmitter 30 and IR emitters 31broadcast data to portable devices 33. The portable devices have both RFand IR receivers. The RF data stream contains encrypted caption text,digitized audio, and control data for one theater 34 or multiple venues.Each theater or venue has one or more IR emitters that send smallamounts of data to synchronize the portable device playback to userposition or the theater presentation. An emitter mounted outside onetheater's entrance may send an identifying code that the portable deviceuses to identify that theater. The device then extracts data from the RFstream for that theater's content, and loads the content into thedevice's memory. Content may include theater name, show times, showsynopses, as well as the text and audio for an upcoming show. When theuser enters the theater, the device receives IR synchronizing codes forthe current show, and the device displays caption text, or play audio,or otherwise synchronize device functions during the show. The IRsynchronizing codes are derived from the show media as describedearlier.

[0029]FIG. 5 illustrates an exemplary embodiment of the basic structureof the data packet stream sent by the high-speed transmitters.

[0030] The figure shows a possible broadcast packet stream of theatershow content for three theaters, along with data packets of ancillaryinformation such as theater show schedules and current date and time.

[0031] The broadcast packet stream contains small groups of data(packets) that contain a small amount of content that will be receivedby the user devices and, if needed, will be loaded into the user deviceslocal memory. The logical state of each user device determines whichpackets are loaded into a particular user device. For example, userdevices that are in theater A would receive and store content fortheater A, but ignore content for theaters B and C. All user deviceswould receive the current time packet, and use the time data to resetthe user device clock. Any user device in front of theater A, B, or Cwould receive and store the theaters' schedule data. The example packetsin the figure show content broken into one-second fragments. So acaptioning device in theater A would receive the caption text packet forthe 1st second of the show, caption text packet for the 2nd second, etc.Each second of content would be loaded into device memory for accessduring the show.

[0032] A large number of individual packet types are possible, eachcontaining different data such as caption text, audio content, videocontent, graphics, images, time, schedules, menus, geographicalinformation, game content, survey questions, advertising, warnings,alerts, etc.

[0033] Each data packet (refer to examples Caption Text, Audio Content,Current Time Packets in FIG. 5) always starts with a unique Header valuethat signifies the start of a new data packet.

[0034] The data packet also contains a Count value that is used toindicate the amount of following data in the packet. The Count is usedto validate the packet and calculate a checksum.

[0035] The Encryption data represents key and authenticationinformation. This, and the Checksum, enables the user device to verifythat following data in the packet is intended for the device and thatthe data has not been corrupted. Corresponding key and authenticationinformation are loaded into the user device.

[0036] The Message Type value defines the content data in the packet.Based on the Message Type, the user device will determine how to decodethe packet data, where to store the data, how and when to use the data.For example, warning text may be immediately displayed; show captiontext may be buffered in device memory until an IR synchronizing codetriggers that caption text display. Menu or schedule information may bebuffered in device memory until the user requests it.

[0037] The Address value designates an identifier for a receivingdevice. This may be a unique address, so the packet is intended for onlyone user device. Or, the address may specify a block of user devices,such as all captioning devices, or all game devices. Or, the Address maydenote the packet as a generic broadcast sent to all devices.

[0038] The optional Start Time value acts as a packet sequence number,but may also be used by the receiving user device to define when thepacket content is used. As packets are received, the user device willbuffer content into its memory in sequence based on start time. As theuser device receives synchronizing codes, it will pull content from thesequential memory or search memory for the content with appropriatestart time. Additional information such as Caption Text Data withFormatting codes, Digitized Audio content, and Current Time and Frameinformation are included in the Data Packet Stream.

[0039] there is a Checksum information that uses a combination of allthe preceding data to verify that the packet has arrived at its locationin its entirety.

[0040]FIG. 6 illustrates an exemplary embodiment of a portable userdevice in accordance with the present invention. All portable deviceshave the common elements of an IR receiver 50, RF receiver 60, IRtransmitter 51, RF transmitter 61, processor 52, memory 54, and powersource 56. Other elements of the devices vary depending on intendedapplications. In this exemplary embodiment, the portable device has botha display 53 for displaying text and/or graphics, in addition to anaudio amplifier 57 for playing sounds, which may be in the form ofspeech or musical tones. Memory 54 is where the data is stored on thedevice. The IR receiver 50 receives a code from the wirelesstransmitter, and processor 52 analyzes the code. The processor 52interprets the code to determine how to handle it. If the code isintended for that device, the device searches its memory 54 for contentcorresponding to the code. The content is then displayed on display 53and/or plays sound through the audio amplifier 57. The portable deviceshown in FIG. 5 also shows user inputs 55, such as pushbuttons, whichmay be used for interactive applications. The high-speed RF transceiversystem could be used to transfer content to the portable device, whilethe IR tranceiver system may be used to precisely aim locationsynchronizing codes to the portable device.

[0041]FIG. 7 is a flowchart that shows how received data packets arehandled by the user device. After the device is powered on, it iscapable of receiving data packets from both IR and RF receivers. Thedevice also monitors user inputs that may come from pushbuttons, touchscreen, speech input, assistive devices, etc. The device alsoperiodically reads its clock, and searches its event database memory forpreloaded events set to trigger at the current time.

[0042] Either receiver may recognize an incoming packet. Any packet thatcontains timing errors, corrupted data, cannot be authenticated, etc.will be ignored. A packet that has correct bit timing, correct header,can be authenticated, can be decrypted, and has a correct checksum willbe marked as valid.

[0043] A valid packet will be compared with the user deviceconfiguration, so that only appropriate packets will be used. Forexample, a user device that displays caption text, but has no audiocapability, may be configured for text only. That device would ignorepackets containing audio content.

[0044] The packet message type determines how the device will use thepacket data. The packet may contain content such as caption text, audio,or video that is buffered in the user device content database memory forfuture use. Other data, such as an urgent text alert message, may besent to the device display immediately. Data such as location and timesynchronizing codes may cause the device to search its content databasememory, extract the content, and present the content. Current time datamay cause the device to reset its internal clock. Event time data willbe placed in the device's content database memory, and will be acted onwhen the internal clock matches the event time. Special effect data maycause the user device to buzz, ring, animate, etc.

[0045] The user device may have some number of user inputs available.These may take the form of pushbuttons, touch-screen soft buttons,touch-screen menus, speech input, assistive device such as abreath-operated puff switch, etc. The user may set preferences, such aspoints of interest, types of movies, age, educational level, etc. Thedevice may use these manually entered preferences, along with learnedpreferences, to alter the presentation of content. For example, achild's device may be set to display captions using a small Englishvocabulary. As the child carries the device from location to location ina museum, the device may also “learn”, or infer, the child's preferencefor a certain type of display.

[0046] The device may also learn a user's direction of travel, andtherefore predict or suggest future destinations. For example, afterwalking down a hallway towards North, the device may alert the user thatthe hallway will end, and a turn to the East or West is necessary.Furthermore, the learned direction, along with any learned and manuallyentered user preferences, may cause the device to deduce a preferencefor an exhibit that is to the East of the hallway end.

[0047] The portable device may automatically infer user preferences fromuse of the device based on the current location of the user device, thetime elapsed at a location, path history (i.e. geographic walkinghistory) of locations visited, and the time elapsed between locations(i.e. walking speed).

[0048] User inputs may also be assigned to device hardware functions.For example, an input pushbutton may cause an IR transmitter to send acode to an adjacent device. Such code may identify the user to anotheruser, or identify the user to an automated device, network, etc. Onecode may cause a door to open, for example, or allow interaction with agame system, etc.

[0049] The user device may present content based on internal logic andits internal clock, without receiving data packets. For example, textcaptions may appear at a predefined time of day to remind a user ofvenue show times, closing time, or remind the user of medication needed,etc. Further, such time events may be logically constrained by thelearned and manually entered user preferences. For example, a restaurantclosing time may not be displayed because the user has walked away fromthe restaurant.

[0050] In closing it is to be understood that the embodiments of theinvention disclosed herein are illustrative of the principals of theinvention. Other modifications may be employed which are within thescope of the invention. Accordingly, the present invention is notlimited to that precisely as is shown and described in the presentspecification.

We claim:
 1. A method for displaying content data on a readable display in conjunction with a media presentation comprising the steps of: displaying media presentation data, said media presentation data including time prompts; said content data having sequences correlated to the time prompts; detecting the time prompts in the media data; and transmitting to the readable display the sequence of content data associated with a detected time prompt.
 2. The method of claim 1 wherein the transmitting is by way of an IR signal.
 3. The method of claim 1 wherein the transmitting is by way of an RF signal.
 4. The method of claim 1 wherein the transmitting is by way of a wired connection.
 5. The method of claim 1 wherein the media data is prerecorded.
 6. The method of claim 1 wherein the media data is a live performance.
 7. The method of claim 1 further including the step of storing the content data in a memory device.
 8. The method of claim 1 wherein the time prompts are optically readable.
 9. A method for presenting content data on at least one user device comprising the steps of: providing content data at a predetermined time; providing time prompts on a media; detecting the time prompts on the media; correlating the content data with the time prompts; and at a given time prompt, transmitting the correlative content data to the user device.
 10. The method of claim 9 wherein the user device has a readable display on which the content data is displayed.
 11. The method of claim 9 wherein the user device has an audio output and the content data is provided to the user through a speaker or headphones.
 12. The method of claim 9 wherein the content data is transmitted to the user device through an IR signal.
 13. The method of claim 9 wherein the content data is transmitted to the user device through an RF signal.
 14. A method for displaying content data on a readable display comprising the steps of: providing content data to be displayed at a predetermined time; providing media presentation data having time prompts; said content data correlated with the time prompts; detecting the time prompts in the media presentation data; at a given time prompts, transmitting the correlative content data to the readable display; and displaying the content data on the readable display.
 15. A method for displaying information on a readable display comprising the steps of: storing information to be displayed at a predetermined time; displaying media data, said media data including time prompts; said information correlated to at least one of the time prompts; and at a given time prompt, transmitting the correlative information to the readable display.
 16. A method for interactive communication in conjunction with a media presentation comprising the steps of: providing a content display device having a readable display; storing content data for display; presenting media presentation data having time prompts; said content data having sequences correlated to the time prompts; detecting the time prompts in the media presentation data; transmitting to the readable display the sequence of content data correlated with a detected time prompt; and providing inputs on the content display device adapted to receive information from a viewer.
 17. An apparatus for streaming digital data to a portable device, during a media presentation including time prompts, comprising: a memory device storing the digital data prior to transmission; a wireless emitter transmitting the digital data; a wireless receiver on the portable device receiving the digital data; a time prompt detector; a data processor associating detected time prompts to the digital data according to predetermined rules; and a transmitter for transmitting digital data correlated with the time prompts from the wireless emitter to the wireless receiver.
 18. A method for presenting content data correlated to a media presentation on at least one user device comprising the steps of: providing content data at a predetermined time; embedding time prompts on a media; presenting the film to at least one user; detecting the time prompts on the media; correlating the content data with the time prompts; transmitting the correlative content data to the user device at a given time prompt; and displaying the content data on the user device.
 19. An apparatus for streaming digital data to a portable device, before and during a media presentation including time prompts, comprising: memory devices for storing the digital data prior to transmission; a wireless emitter sending the synchronizing digital data; a distribution data server to send media presentation content a wireless transmitter transmitting the streaming digital data; a transmitter for transmitting digital data containing multiple media contents; a time prompt detector; a wireless receiver on the portable device receiving the digital data; a data processor associating detected time prompts to the digital data according to predetermined rules; and the transmitting digital data being correlated with the time prompts from the wireless emitter to the wireless receiver.
 20. A portable device that automatically determines a user preference based on the location of the device, the time elapsed at the location, and the history of visited locations, the device comprising: a receiver for receiving a code from a remote transmitter, said code being a location code corresponding to the remote transmitter; a timer for determining the time elapsed at a location and in between a plurality locations; and a memory for storing the code corresponding to each of the locations visited by the user of the device.
 21. A portable device that automatically predicts a user's destination location and presents media content for that location, the portable device comprising: an interface for inputting a user preference; a receiver for receiving a code from a remote transmitter, said code being a location code corresponding to the remote transmitter; a timer for determining the time elapsed at a location and in between locations; a memory for storing the code corresponding to each the locations visited by the user of the device. 